from sklearn_benchmarks.reporting.hp_match import HPMatchReporting

scikit-learn vs. ONNX Runtime

reporting = HPMatchReporting("onnx", config="config.yml")
reporting.run()

KNeighborsClassifier_brute_force

onnx (1.9.0) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=brute.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_brute_force	predict	100000	1000	100	15.414	0.039	0.0	0.015	-1	100	0.842	2.970	0.039	5.190	0.069	See	See
1	KNeighborsClassifier_brute_force	predict	100000	1000	100	15.414	0.039	0.0	0.015	-1	100	0.842	0.024	0.003	651.320	69.111	See	See
2	KNeighborsClassifier_brute_force	predict	100000	1	100	0.297	0.007	0.0	0.297	-1	100	1.000	2.970	0.039	0.100	0.003	See	See
3	KNeighborsClassifier_brute_force	predict	100000	1	100	0.297	0.007	0.0	0.297	-1	100	1.000	0.024	0.003	12.567	1.370	See	See
4	KNeighborsClassifier_brute_force	predict	100000	1000	100	15.381	0.150	0.0	0.015	-1	5	0.724	2.974	0.042	5.172	0.089	See	See
5	KNeighborsClassifier_brute_force	predict	100000	1000	100	15.381	0.150	0.0	0.015	-1	5	0.724	0.023	0.002	668.938	56.326	See	See
6	KNeighborsClassifier_brute_force	predict	100000	1	100	0.301	0.006	0.0	0.301	-1	5	1.000	2.974	0.042	0.101	0.002	See	See
7	KNeighborsClassifier_brute_force	predict	100000	1	100	0.301	0.006	0.0	0.301	-1	5	1.000	0.023	0.002	13.093	1.124	See	See
8	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.059	0.039	0.0	0.017	-1	1	0.650	1.990	0.038	8.573	0.166	See	See
9	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.059	0.039	0.0	0.017	-1	1	0.650	0.022	0.001	787.716	49.368	See	See
10	KNeighborsClassifier_brute_force	predict	100000	1	100	0.301	0.008	0.0	0.301	-1	1	1.000	1.990	0.038	0.151	0.005	See	See
11	KNeighborsClassifier_brute_force	predict	100000	1	100	0.301	0.008	0.0	0.301	-1	1	1.000	0.022	0.001	13.900	0.943	See	See
12	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.104	0.162	0.0	0.017	1	5	0.724	2.178	0.008	7.851	0.080	See	See
13	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.104	0.162	0.0	0.017	1	5	0.724	0.022	0.005	791.237	200.878	See	See
14	KNeighborsClassifier_brute_force	predict	100000	1	100	0.304	0.007	0.0	0.304	1	5	1.000	2.178	0.008	0.139	0.003	See	See
15	KNeighborsClassifier_brute_force	predict	100000	1	100	0.304	0.007	0.0	0.304	1	5	1.000	0.022	0.005	14.052	3.578	See	See
16	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.342	0.037	0.0	0.017	1	1	0.650	1.184	0.005	14.648	0.070	See	See
17	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.342	0.037	0.0	0.017	1	1	0.650	0.019	0.000	917.289	13.479	See	See
18	KNeighborsClassifier_brute_force	predict	100000	1	100	0.300	0.004	0.0	0.300	1	1	1.000	1.184	0.005	0.253	0.004	See	See
19	KNeighborsClassifier_brute_force	predict	100000	1	100	0.300	0.004	0.0	0.300	1	1	1.000	0.019	0.000	15.845	0.328	See	See
20	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.355	0.003	0.0	0.017	1	100	0.842	2.185	0.007	7.944	0.026	See	See
21	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.355	0.003	0.0	0.017	1	100	0.842	0.019	0.000	903.311	5.179	See	See
22	KNeighborsClassifier_brute_force	predict	100000	1	100	0.303	0.003	0.0	0.303	1	100	1.000	2.185	0.007	0.139	0.001	See	See
23	KNeighborsClassifier_brute_force	predict	100000	1	100	0.303	0.003	0.0	0.303	1	100	1.000	0.019	0.000	15.752	0.177	See	See
24	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.611	0.156	0.0	0.004	-1	100	0.886	2.824	0.046	1.279	0.059	See	See
25	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.611	0.156	0.0	0.004	-1	100	0.886	0.011	0.004	334.902	133.473	See	See
26	KNeighborsClassifier_brute_force	predict	100000	1	2	0.235	0.004	0.0	0.235	-1	100	1.000	2.824	0.046	0.083	0.002	See	See
27	KNeighborsClassifier_brute_force	predict	100000	1	2	0.235	0.004	0.0	0.235	-1	100	1.000	0.011	0.004	21.793	8.644	See	See
28	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.671	0.015	0.0	0.004	-1	5	0.878	2.875	0.054	1.277	0.025	See	See
29	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.671	0.015	0.0	0.004	-1	5	0.878	0.009	0.002	386.845	81.610	See	See
30	KNeighborsClassifier_brute_force	predict	100000	1	2	0.235	0.004	0.0	0.235	-1	5	1.000	2.875	0.054	0.082	0.002	See	See
31	KNeighborsClassifier_brute_force	predict	100000	1	2	0.235	0.004	0.0	0.235	-1	5	1.000	0.009	0.002	24.728	5.232	See	See
32	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.660	0.029	0.0	0.004	-1	1	0.850	1.848	0.023	1.981	0.030	See	See
33	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.660	0.029	0.0	0.004	-1	1	0.850	0.009	0.002	418.761	96.193	See	See
34	KNeighborsClassifier_brute_force	predict	100000	1	2	0.235	0.005	0.0	0.235	-1	1	1.000	1.848	0.023	0.127	0.003	See	See
35	KNeighborsClassifier_brute_force	predict	100000	1	2	0.235	0.005	0.0	0.235	-1	1	1.000	0.009	0.002	26.897	6.198	See	See
36	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.687	0.009	0.0	0.004	1	5	0.878	2.002	0.009	1.842	0.009	See	See
37	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.687	0.009	0.0	0.004	1	5	0.878	0.003	0.000	1302.299	83.828	See	See
38	KNeighborsClassifier_brute_force	predict	100000	1	2	0.234	0.003	0.0	0.234	1	5	1.000	2.002	0.009	0.117	0.002	See	See
39	KNeighborsClassifier_brute_force	predict	100000	1	2	0.234	0.003	0.0	0.234	1	5	1.000	0.003	0.000	82.684	5.451	See	See
40	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.675	0.013	0.0	0.004	1	1	0.850	1.104	0.010	3.330	0.032	See	See
41	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.675	0.013	0.0	0.004	1	1	0.850	0.002	0.000	2189.713	311.154	See	See
42	KNeighborsClassifier_brute_force	predict	100000	1	2	0.206	0.004	0.0	0.206	1	1	1.000	1.104	0.010	0.186	0.004	See	See
43	KNeighborsClassifier_brute_force	predict	100000	1	2	0.206	0.004	0.0	0.206	1	1	1.000	0.002	0.000	122.456	17.529	See	See
44	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.331	0.016	0.0	0.003	1	100	0.886	1.892	0.010	1.760	0.013	See	See
45	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.331	0.016	0.0	0.003	1	100	0.886	0.003	0.000	1240.632	100.848	See	See
46	KNeighborsClassifier_brute_force	predict	100000	1	2	0.207	0.004	0.0	0.207	1	100	1.000	1.892	0.010	0.109	0.002	See	See
47	KNeighborsClassifier_brute_force	predict	100000	1	2	0.207	0.004	0.0	0.207	1	100	1.000	0.003	0.000	76.953	6.392	See	See

KNeighborsClassifier_kd_tree

onnx (1.9.0) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=kd_tree.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	126.699	0.000	0.0	0.127	-1	1	0.956	0.522	0.021	242.797	9.991	See	See
1	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	126.699	0.000	0.0	0.127	-1	1	0.956	0.803	2.400	157.687	470.946	See	See
2	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.541	0.091	0.0	2.541	-1	1	1.000	0.522	0.021	4.870	0.266	See	See
3	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.541	0.091	0.0	2.541	-1	1	1.000	0.803	2.400	3.163	9.447	See	See
4	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	120.391	0.000	0.0	0.120	1	1	0.956	0.889	0.009	135.440	1.314	See	See
5	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	120.391	0.000	0.0	0.120	1	1	0.956	0.002	0.001	70112.657	21209.373	See	See
6	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.269	0.127	0.0	2.269	1	1	1.000	0.889	0.009	2.553	0.145	See	See
7	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.269	0.127	0.0	2.269	1	1	1.000	0.002	0.001	1321.536	406.514	See	See
8	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	125.894	0.000	0.0	0.126	-1	5	0.978	0.967	0.010	130.233	1.323	See	See
9	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	125.894	0.000	0.0	0.126	-1	5	0.978	0.005	0.001	24096.853	4903.738	See	See
10	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.546	0.104	0.0	2.546	-1	5	1.000	0.967	0.010	2.634	0.111	See	See
11	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.546	0.104	0.0	2.546	-1	5	1.000	0.005	0.001	487.275	101.133	See	See
12	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	142.829	0.000	0.0	0.143	1	5	0.978	1.722	0.021	82.963	1.012	See	See
13	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	142.829	0.000	0.0	0.143	1	5	0.978	0.003	0.001	46358.164	14284.711	See	See
14	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.546	0.108	0.0	2.546	1	5	1.000	1.722	0.021	1.479	0.065	See	See
15	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.546	0.108	0.0	2.546	1	5	1.000	0.003	0.001	826.326	257.041	See	See
16	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	142.740	0.000	0.0	0.143	1	100	0.978	5.799	0.028	24.616	0.118	See	See
17	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	142.740	0.000	0.0	0.143	1	100	0.978	0.010	0.001	13941.002	1709.072	See	See
18	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.538	0.059	0.0	2.538	1	100	1.000	5.799	0.028	0.438	0.010	See	See
19	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.538	0.059	0.0	2.538	1	100	1.000	0.010	0.001	247.898	30.926	See	See
20	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	146.033	0.000	0.0	0.146	-1	100	0.978	3.286	0.026	44.436	0.351	See	See
21	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	146.033	0.000	0.0	0.146	-1	100	0.978	0.014	0.002	10221.972	1326.144	See	See
22	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.567	0.125	0.0	2.567	-1	100	1.000	3.286	0.026	0.781	0.038	See	See
23	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.567	0.125	0.0	2.567	-1	100	1.000	0.014	0.002	179.699	24.890	See	See
24	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.000	0.0	0.000	-1	1	0.894	0.021	0.001	1.817	0.051	See	See
25	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.000	0.0	0.000	-1	1	0.894	0.002	0.000	18.020	3.591	See	See
26	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	1	1.000	0.021	0.001	0.242	0.010	See	See
27	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	1	1.000	0.002	0.000	2.399	0.483	See	See
28	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.000	0.0	0.000	1	1	0.894	0.018	0.000	2.100	0.032	See	See
29	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.000	0.0	0.000	1	1	0.894	0.001	0.000	64.443	17.937	See	See
30	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	1	1.000	0.018	0.000	0.283	0.015	See	See
31	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	1	1.000	0.001	0.000	8.670	2.453	See	See
32	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.039	0.000	0.0	0.000	-1	5	0.923	0.022	0.001	1.811	0.104	See	See
33	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.039	0.000	0.0	0.000	-1	5	0.923	0.002	0.000	18.958	4.073	See	See
34	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	5	1.000	0.022	0.001	0.239	0.017	See	See
35	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	5	1.000	0.002	0.000	2.506	0.549	See	See
36	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.039	0.000	0.0	0.000	1	5	0.923	0.019	0.000	2.013	0.028	See	See
37	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.039	0.000	0.0	0.000	1	5	0.923	0.001	0.000	64.577	17.894	See	See
38	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	5	1.000	0.019	0.000	0.265	0.009	See	See
39	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	5	1.000	0.001	0.000	8.487	2.365	See	See
40	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.062	0.000	0.0	0.000	1	100	0.901	0.033	0.000	1.875	0.014	See	See
41	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.062	0.000	0.0	0.000	1	100	0.901	0.001	0.000	99.694	28.102	See	See
42	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	100	1.000	0.033	0.000	0.159	0.012	See	See
43	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	100	1.000	0.001	0.000	8.468	2.468	See	See
44	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.061	0.000	0.0	0.000	-1	100	0.901	0.035	0.000	1.748	0.018	See	See
45	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.061	0.000	0.0	0.000	-1	100	0.901	0.002	0.000	27.237	3.874	See	See
46	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	100	1.000	0.035	0.000	0.149	0.005	See	See
47	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	100	1.000	0.002	0.000	2.321	0.338	See	See

HistGradientBoostingClassifier_best

onnx (1.9.0) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: learning_rate=0.01, n_iter_no_change=10.0, max_leaf_nodes=100.0, max_bins=255.0, min_samples_leaf=100.0, max_iter=300.0.

	estimator	function	n_samples_train	n_samples	n_features	n_iter	mean_duration	std_duration	iteration_throughput	latency	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	HistGradientBoostingClassifier_best	predict	100000	1000	100	300	0.476	0.014	0.002	0.000	0.804	0.107	0.000	4.437	0.132	See	See
1	HistGradientBoostingClassifier_best	predict	100000	1000	100	300	0.476	0.014	0.002	0.000	0.804	0.019	0.001	25.566	1.835	See	See
2	HistGradientBoostingClassifier_best	predict	100000	1	100	300	0.392	0.009	0.000	0.392	1.000	0.107	0.000	3.652	0.089	See	See
3	HistGradientBoostingClassifier_best	predict	100000	1	100	300	0.392	0.009	0.000	0.392	1.000	0.019	0.001	21.039	1.467	See	See